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INSTRUCTIONAL  STRATEGIES 


USING  LOW-COST  SIMULATION  FOR  ELECTRONIC  MAINTENANCE 


SUMMARY  AND  C«)HCLUSIONS 


MILITARY  PROBLEM 


The  U.S.  Amy  Air  Defense  School  has  the  responsibility  of  training 
men  to  maintain  complex  electronic  equipment.  Due  to  the  critical 
nature  of  systems  such  as  the  Improved  Hawk,  there  Is  limited  access  to 
equipment.  Therefore,  there  Is  a r ^ed  to  develop  training  alternatives 
which  require  a minimum  amount  of  time  on  the  actual  equipment. 


RESEARCH  OBJECTIVE 


The  research  project  was  to  determine  whether  low-cost  simulation 
(l.e.,  photos  and  slides)  used  In  conjunction  with  carefully  engineered 
Instructional  strategies  could  Improve  training  In  maintenance  of  Air 
Defense  Systems  when  administered  by  U.S.  Army  Air  Defense  School  (USAADS) 
Instructors. 


METHOD 


Lesson  plans  and  simulation  materials  were  developed  for  five  blocks 
of  Instruction  on  the  Improved  Hawk  IHIPIR.  These  were  used  In  two 
experimental  classes  and  the  results  were  compared  with  two  control  group 
classes. 


RESULTS 


The  experimental  classes  demonstrated  superior  performance  Involving 
content  of  the  five  blocks  on  several  measures: 

1)  Successfully  completing  troubleshooting  problems 
(simulated  on  equipment  turned  off)  In  the  tine 
allowed,  with  significantly  fewer  errors. 


Ill 


2)  Denonstratlng  checks  and  adjustments  faster,  and 
with  fewer  errors; 

3)  Better  scores  on  quizzes  which  Involved  performance- 
oriented  Items. 


CONCLUSIONS 


The  Instructional  strategies  and  simulation  methods  significantly 
Improved  training  where  real  equipment  was  In  short  supply,  within  the 
practical  constraints  of  the  USAADS  situation  and  with  minimal  Increase 
In  costs. 
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BACKGROUND 


MILITARY  PROBLEM 


The  Army  must  train  men  to  maintain  complex  electronic  equipment, 
with  stringently  limited  time  for  practice  on  the  real  equipment.  The 
training  problems  are  especially  severe  with  newly  developed  and  complex 
equipment.  One  example  of  a critical  system  Is  the  Improved  Hawk  System. 
Complex  electronic  simulators  might  provide  the  needed  practice,  but 
their  cost  and  development  time  make  this  approach  generally  Infeasible 
and  unacceptable.  Another  constraint  for  the  present  research  Is  that 
training  must  be  accomplished  In  existing  facilities  by  existing  staff 
without  extensive  special  training. 


RELEVANT  STUDIES  IN  THE  LITERATURE 


One  promising  approach  to  the  lack  of  time  on  actual  equipment  Is 
the  use  of  v9ry  simple,  low-cost  trainers  or  mock-ups  having  only  gross 
visual  fidelity;  for  example,  devices  having  dummy  Instruments,  photos, 
or  even  line  drawings.  Studies  of  a variety  of  procedural  tasks  have 
demonstrated  the  effectiveness  of  such  devices  (Cox,  e^.  al . , 1965; 
Denenberg,  195A;  Dougherty,  at.  ^. , 1957;  Fox,  al . , 1969;  Grlmsley, 
1969,  a,  b,  c;  Miller,  1958;  Prophet  and  Boyd,  1970).  However,  one 
should  not  presume  that  such  slmrle  devices  would  be  effective  for  all 
types  of  training  or  In  all  training  situations.  As  training  becomes 
more  advanced,  the  students  become  Increasingly  familiar  with  the  equip- 
ment, and  therefore,  may  no  lo.iger  need  visual  reference  to  the  equipment. 
Also,  there  are  Increasingly  complex  logical  Inferences  (cognitive  re- 
quirements) - Involved  In  troubleshooting,  so  perhaps  there  Is  relatively 
leas  need  to  establish  visualization  processes,  even  though  visual  cues 
continue  to  be  Involved  In  the  task. 

Swanson  (1954) , using  highly  experienced  electronic  maintenance 
technicians  as  subjects,  found  no  differences  among  training  aids  with 
various  levels  of  realism.  These  technicians  could  perform  effectively 
from  general  verbal  directions  or  abstract  symbols,  as  they  had  been 
trained  to  do.  In  a follow-up  experiment,  Swanson,  Lumsdalne,  and  Aukes 
(1956),  using  Inexperienced  airmen  as  subjects,  found  that  training  with 
a visual  mock-up  did  lead  to  superior  performance  on  a parts  recognition 
test,  as  compared  with  those  who  had  similar  training  without  the  visual 
mock-up.  One  would  expect,  therefore,  that  the  effectiveness  of  visual 
mock-ups  would  depend  upon  the  stage  of  training  and  the  kind  of  per- 
formance tested. 
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In  previous  ARI-sponsored  HumRRO  research*,  the  Instructional  systee 
for  conventional  Hawk  training  was  analyzed  for  Influences  on  instructor 
performance.  Although  guided  by  the  Program  of  Instruction  (POI) , lesson 
plans  and  vault  files,  the  Instructor  has  considerable  latitude  in  his 
instructional  approach,  especially  if  he  is  a Primary  Instructor.  The 
system  further  tends  to  give  more  prestige  to  the  platform  instructor 
than  to  the  practical  exercise  instructor.  There  is  also  little  incen- 
tive for  an  Instructor  to  disseminate  apparently  effective  instructional 
strategies  to  others.  There  appears,  therefore,  to  be  a definite  need 
for  preparing  Instructors  in  performance-oriented  Instruction. 

As  part  of  INTERFACE,  an  experimental  Instructor  Workshop  in  Func- 
tional Context  Training  was  conducted.  In  the  Workshop,  it  appeared  that 
instructors  could  learn  to  use  a photo  mock-up  with  skill  and  ingenuity, 
although  at  first  they  seemed  to  have  considerable  difficulty  in  faith- 
fully following  a Job  sequence  in  their  Instruction. 


RESEARCH  OBJECTIVE 


The  research  reported  here  was  designed  to  answer  the  following 
question: 

Can  low-cost  simulation  (l.e.,  photos  and  slides)  Improve 
instruction  in  electronic  maintenance  on  complex  systems, 
such  as  Improved  Hawk  (in  which  real  equipment  is  scarce) , 
when  used  by  typical  USAADS  instructors  following  carefully 
engineered  instructional  strategies? 


*HumRRO  Work  Unit  INTERFACE 
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METHOD 


SUBJECTS 


• The  subjects  were  78  students  enrolled  in  four  successive  classes 
of  the  Improved  Hawk  Firing  Section  Mechanic  (2AC20)  Program  of  Instruc- 
tion (POI) . The  POI  covers:  (1)  the  Launcher,  and  (2)  the  Improved 

High  Power  Illumination  Radar  (IHIPIR) . The  first  two  classes  (75-5, 

N-23  and  75-6,  N»16)  were  pooled  to  form  the  control  group,  and  the  other 
two  classes  (75-7,  N«22  and  75-8,  N«17)  were  pooled  to  form  the  experi- 
mental group. 


BLOCKS  OF  INSTRUCTION  (LESSONS) 


The  research  investigated  effects  of  the  methods  applied  to  the 
following  five  blocks  of  instruction  on  the  Improved  High  Power  Illumi- 
nation Radar,  IHIPIR  (presented  immediately  after  the  Introduction): 

1.  Energizing  and  Control  Circuits,  8 periods*  of 
classroom  activity. 

2.  Energizing  and  Control  Circuits,  6 periods  of 
practice  on  station  with  the  equipment. 

3.  AC  and  DC  Power  Distribution,  4 periods  of 
classroom  activity. 

4.  AC  and  DC  Power  Distribution,  12  periods  of 
practice  with  the  equipment  on  station. 

5.  Transmitter  RF  Generation,  Arc  Detection,  and 

- ' Modulator  Bias  Circuits,  8 periods  of  classroom 
activity. 

During  the  first  four  blocks  of  instruction  the  student  is  learning 
appearance  and  location  of  various  components  while  performing  the  basic 
operations  Involved  in  energizing  the  equipment,  following  the  basic 
conventions  of  their  technical  manuals.  Only  the  material  in  the  last 
block  of  instruction  is  considered  difficult  to  understand,  compared 
with  typical  blocks  of  Instruction.  The  classroom  activity  is  designated 


*A  period  is  45  minutes  with  a 10  minute  break  between  periods. 
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"PE  3"  which  stands  for  "Practical  Exercise,  type  3",  signifying  the 
Intention  of  USAADS  to  make  It  as  much  like  a Job  activity  as  possible, 
rather  than  like  the  traditional  lecture. 


TREATMENTS 


Control  Group 


This  section  describes  the  general  approach  In  the  conventional 
classes,  as  a basis  for  comparison  with  the  experimental  treatments 
that  will  be  described  In  the  next  sections. 


Classroom  (PE  3).  The  classroom  work  for  the  Control  Group  during 
the  three  PE  3 blocks  Involved  theory,  with  large  amounts  of  circuit- 
tracing using  schematic  diagrams.  Circuit-tracing  typically  moved  left- 
to-right  as  the  instructor  explained  how  the  circuit  "works".  Various 
facts  were  mentioned  as  they  applied  to  particular  points.  There  were 
few  If  any  visual  aids  beyond  blackboard  and  chalk  and  schematic  diagrams. 
Prior  to  this,  the  students  have  had  six  periods  with  the  equipment  (with 
only  a part  of  that  time  spent  close  enough  to  the  radar  to  become  fami- 
liar with  it).  The  average  student  had  little  more  experience  with  the 
radar  than  the  majority  of  readers  of  this  report.  Procedures  were  not 
described  during  these  periods;  rather,  the  "doing"  was  relegated  to  the 
work  on  station. 


On  Station  (PE  1).  During  the  two  PE  1 blocks  the  students  learned 
to  perform  checks  and  adjustments  and  fault  Isolation.  The  Primary 
Instructor  was  assisted  by  three  other  instructors  (each  Instructor  had 
about  five  students  working  with  one  radar).  Each  Instructor  works  with 
one  pair -of  students  at  a time:  one  student  reads  the  procedure  from 

the  manual  while  the  other  does  It.  The  other  students  were  generally 
supposed  to  observe  the  practice.  Sometimes  the  students  not  performing 
on  the  radar  were  Instructed  to  perform  various  supplementary  tasks  at 
their  tables  while  waiting  for  their  turn  on  the  equipment.  It  appeared 
difficult  for  the  Instructors  to  devise  really  worthwhile  activities  for 
those  not  on  the  radar,  since  the  students  on  the  equipment  had  to  be 
monitored  closely  for  safety  of  personnel  and  equipment. 
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Experimental  Group;  Instructional  Strategies 


For  the  experimental  classes  new  instructional  strategies  were 
devised.  An  Instructional  strategy  Is  a general  rule  for  presenting 
content  Information  so  that  it  Is  easier  to  learn  and  apply  on  the  Job. 
These  rules  cover  sequencing  of  Instruction  and  various  other  practices. 
Each  Instructional  strategy  tends  to  be  especially  appropriate  for 
particular  purposes  and  for  particular  kinds  of  Information  (e.g.,  In 
learning  how  to  do  a task,  It  Is  generally  efficient  to  practice  the 
steps  In  the  task  sequence) . 

Instructional  strategies  aic  commonly  referred  to  as  "presenting 
things  In  a logical  order".  Although  that  Is  true,  It  Is  hardly  spe- 
cific enough  to  be  of  much  help  In  designing  Instruction. 

Instructional  strategies  are  often  used  In  various  combinations. 

All  the  Instructional  strategies  In  this  study  use  visual  simulation. 

In  combination  with  other  strategies.  Therefore,  the  effects  of  the 
visuals  were  confounded  with  the  other  instructional  strategies,  so 
their  independent  effects  can  not  be  determined. 


Visual  Simulation 


Classroom  (PE  3).  The  training  equipment  for  the  experimental 
classes  consisted  of  two  large-scale  color  photos  (60"  x 36"),  one  for 
each  end  of  the  radar,  and  a series  of  color  slides.  The  two  ends  of 
the  radar  (depleted  in  Figure  1)  contained  almost  all  of  the  relevant 
displays  and  controls  (although  sometimes  these  were  located  behind  a 
panel).  As  a general  practice,  the  Instructors  were  to  begin  with  the 
large  color  photos  for  orientation  and  then  refer  to  the  slides  for 
details  on  each  particular  teaching  point. 

The  large  color  photos  were  hung  at  the  f..ont  of  the  room  In  front 
of  the  chalkboard*,  and  were  removed  when  necessary  to  use  the  black- 
board. Before  class  75-8,  In  order  to  facilitate  access  to  the  black- 
board, they  were  mounted**  on  cork  panels  which  slide  In  from  either 
side.  This  arrangement  seemed  satisfactory  In  all  respects. 


*To  avoid  glare  from  celling  lights,  the  panels  were  tilted  out 
(about  6")  at  the  top. 

**Mounting  by  a cord  facilitated  removal  and,  storage. 
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For  the  unit  on  AC  and  DC  Power  Supplies  there  were  color  photos 
(10"  X 12")  of  each  Power  Supply.  These  could  be  hung  on  the  large 
photos  In  the  proper  locations  by  means  of  small  screw  heads  protruding 
from  the  frame. 


On  Station  (PE  1).  For  each  pair  of  students  there  was  a set  of  two 
color  photos  like  those  used  In  the  classroom  by  the  Instructor,  but  on 
a somewhat  smaller  scale  (40"  x 24"  each).  These  were  hung  by  pegboard 
hooks  around  the  perimeter  of  each  station.  To  represent  the  Internal 
components  there  was  a looseleaf  notebook  provided  for  each  student  with 
line  drawings  of  the  equipment  located  behind  each  of  the  panels,  numeri- 
cally coded  to  the  panels.  (Students  tended  to  keep  these  notebooks 
available  because  the  looseleaf  covers  were  also  handy  to-  hold  their  check 
manuals.)  Synchronized  tape/sllde  equipment  was  also. available. 


Structuring  the  Leasons 


The  Instructional  strategies  concern  the  structure  Inherent  In  the 
lessons.  The  structure  Is  most  apparent  In  the  sequence  of  presentation, 
although  It  may  be  also  a matter  of  emphasis  and  other  aspects  of  style 
of  presentation.  In  the  Instructional  blocks  studied,  the  most  Important 
structures  to  be  considered  are  the  following: 


Job  Situation  and  Job  Sequence.  This  Is  a matter  of  presenting 
Information  as  It  Is  to  be  used  on  the  job,  most  notably  In  the  sequence 
of  job  performance.  (However,  It  may  be  only  a segment  of  a task  to  be 
performed.)  This  Is  often  called  the  "functional  context"  method  of 
presentation. 

The  job  sequence  In  classroom  presentations  (along  with  using  visual 
simulation)  was  used  extensively.  This  was  In  contrast  to  the  conven- 
tional lesson  plans,  which  generally  discussed  the  circuits  In  the  sequence 
they  appeared  In  the  manuals. 

The  most  common  use  of  the  "job  situation"  strategy  was  In  discussing 
the  functional  diagrams.  First,  the  checks  and  adjustments  would  be 
demonstrated  and  discussed,  using  the  color  photos  and  slides,  showing 
how  a trouble  would  lead  to  the  Fault  Isolation  Procedures  (FIP).  Then 
the  FIP  would  be  demonstrated  and  discussed,  showing  how  that  delineates 
a circuit  In  the  schematics  within  which  the  fault  must  lie.  Then  the 
circuit  Is  traced  and  discussed,  but  (unlike  the  conventional  treatment) 

In  the  context  of  looking  for  a trouble. 
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Identify  End  Points  First  in  Schematics.  This  is  a matter  of  locating 
the  first  and  last  functional  reference  points  In  the  schematics  before 
tracing  the  rest  of  the  circuit.  Thus,  the  goal  of  the  tracing  Is  Identi- 
fied first. 

Relating  Elements  in  Schematic  Diagrams  to  the  Equipment.  In  the 
above  strategy,  noting  end  points  also  served  to  relate  the  schematics 
to  the  real  equipment,  since  the  end  point  almost  always  was  observable 
on  the  equipment  (e.g.,  a meter).  The  slides  also  provided  opportunity 
for  relating  points  In  the  schematics  to  their  visual  appearance. 

Grouping.  This  Is  the  strategy  of  sorting  the  subject  matter  Into 
natural  categories.  It  is  commonly  called  the  "whole-to-part"  method. 

In  introducing  the  general  categories.  It  Is  Important  to  avoid  vagueness. 

In  the  psychological  literature  such  grouping  is  called  "chunking" 
or  "advance  organizers".  The  human  mind  can  only  think  about  a few  items 
at  once  (this  is  the  limit  of  immediate  memory,  or  what  a person  Is  con- 
sciously aware  of  at  any  particular  moment).  It  seems  that  a person  can 
only  hold  about  seven  Items  (technically  called  "chunks"  of  Information) 

In  his  Immediate  memory  at  one  time  (G.  A.  Miller,  1956).  For  Instance, 

If  a person  looks  up  an  unfamiliar  telephone  number  (7  digits  or  "chunks" 
of  Information),  he  can  probably  remember  it  until  he  dials  It;  but  if 
someone  Interrupts  him  by  asking  for  some  other  information,  that  seems 
to  "push"  some  of  the  digits  out  of  his  memory  and  he  will  probably  have 
to  look  It  up  again. 

The  Implication  for  instructional  practice  Is  that  the  general  cate- 
gories (the  "whole")  should  be  subdivided  In  stages,  with  fewer  than 
seven  subdivisions  at  any  particular  stage  (generally  three  to  five  sub- 
divisions are  effective).  A common  mistake  Ln  applying  the  whole-to-part 
strategy  Is  to  jump  from  the  general  topic  down  to  the  detailed  level, 
so  that  the  learner  can  not  relate  the  two  levels. 

A good- example  of  the  "grouping"  strategy  occurs  In  the  first  experi- 
mental block  of  instruction.  Step  3 of  the  FIF  concerns  various  events 
that  might  prevent  the  radar  from  going  Into  "Standby".  This  generally 
Is  a matter  of  an  Interlock  switch  being  open  or  faulty.  This  Is  not 
apparent  to  the  novice  until  he  gets  well  Into  the  14  substeps  of  Step  3, 
which  are  spread  over  two  large  (11"  x 16-1/2")  pages  of  the  FIP.  These 
Interlock  switches  are  ''.hecked  out  with  a selector  switch  and  Indicator 
light  at  each  side  of  the  radar.  The  first  cluster  (or  group)  of  substeps 
Is  Step  3a  In  which  the  Indicator  lights  themselves  are  checked  to  be  sure 
that  they  will  give  the  proper  Indications.  The  second  cluster  covers 
Steps  3b  through  3g  In  which  the  Interlocks  on  the  Transmitter  side  are 
checked  out  by  a similar  switch.  The  third  cluster  Includes  Steps  3h 
through  3k  In  which  the  Interlocks  on  the  Radar  Set  Group  (RSG)  side  are 
checked  out  by  a similar  switch.  The  last  cluster,  Steps  3L  through  3n, 
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covers  the  remaining  components  that  might  cause  trouble  between  the 
Interlocks  and  the  "Standby"  Indicator  lights,  Including  relays  and  a 
holding  circuit.  The  students  were  directed  to  mark  these  clusters  of 
steps  In  their  manuals.  Note  that  the  grouping  is  not  arbitrary,  and 
that  It  corresponds  precisely  to  visual  aspects  of  the  equipment  (the 
switches  and  Indicator  lights). 


Basic  Case  and  Variations,  It  Is  often  easier  to  remember  several 
similar  procedures  If  they  are  thought  of  simply  as  variations  of  one 
basic  method.  That  way  the  person  has  many  fewer  Items  to  learn.  When 
the  variations  differ  In  complexity,  the  simplest  is  generally  taught 
first  (the  so  called  "simple-to-complex"  strategy). 

This  is  illustrated  in  the  above  example  of  clustering  substeps 
within  the  second  and  third  clusters.  The  substeps  in  each  of  these 
clusters  is  identical  in  preparation  and  in  holding  the  Standby  button 
depressed,  differing  only  in  the  particular  position  of  the  selector 
switch. 


Causal  Sequence,  With  this  strategy  a cause-and-effect  chain  of 
events  is  described  in  sequence.  Note  that  this  is  not  necessarily  the 
same  as  a time  sequence  because  the  events  may  occur  simultaneously  or 
contlnuoufaly  (e.g.,  in  a system  of  mechanical  linkages  the  whole  set 
moves  together,  although  each  link  causes  movement  in  the  next  one  it 
touches,  going  from  input  to  output  motion). 


Left-to-Right.  This  was  not  one  of  the  experimental  instructional 
strategies  but  is  mentioned  because  it  is  perhaps  the  most  commonly 
encountered  arbitrary  strategy.  With  this  strategy  the  items  are  described 
in  a sequence  moving  from  lef t-to-rlght,  as  they  occur.  For  Instance, 
an  Instructor  may  describe  the  Indicators  and  controls  on  an  item  of 
equipment  "starting  from  the  upper  left-hand  comer  and  moving  across 
the  panel".  The  strong  "lef t-to-rlght"  tendency  is  based  upon  people's 
reading  habits.  Although  it  is  sometimes  appropriate  (e.g.,  reading 
schematics),  it  very  often  is  not.  Therefore,  whenever  one  encounters 
the  left-to-right  sequence  in  instruction,  one  should  analyze  the  sit- 
uation to  determine  whether  it  Is  an  appropriate  strategy. 


Experimental  Group  Instruction 

Classroom  (PE  S) 


Energizing  and  Control  Circuits  (8  Periods),  Here  the  predominant 
instructional  strategy  was  the  Job  sequence  of  checks  and  adjustments 


and  Fault  Isolation  Procedures  (FIP) , Illustrated  with  the  large  color 
photos  and  a series  of  slides.  This  began  with  a short  series  of  checks 
and  adjustments  (Steps  1-3,  Table  3-8  in  TM  9-1430-533-12-1),  Then  each 
step  was  repeated,  assuming  there  was  a fault,  which  led' to  a correspond- 
ing step  in  the  FIP,  and  using  the  functional  schematic  as  specified. 

In  Step  3 of  the  FIP  the  instructional  strategy  of  grouping  the  substeps 
was  also  employed  (see  the  lesson  plan  in  Appendix  A,  Vol.  II,  for  more 
details  of  the  presentation).'*^ 

In  order  to  further  assist  students  in  finding  components  of  the 
equipment,  two  locator  diagrams,  developed  by  Raytheon,  were  handed  out. 
Using  these  diagrams,  the  student  could  locate  an  item  from  knotting  either 
the  title  of  a block  on  the  schematics,  or  the  numerical  code  that  appears 
by  that  title. 

The  instructor  seemed  to  have  little  difficulty  in  adopting  the 
methods.  In  his  presentation  he  referred  repeatedly  to  the  color  photos 
and  to  the  slides,  as  required  by  the  lesson  plan.  There  did  not  appear 
to  be  any  tendency  to  "taper  off"  and  return  to  traditional  methods. 


AC  and  DC  Powev  Dietribution  (4  Pern-oda),  The  prominent  instructional 
strategy  here  was  grouping  of  power  supplies  supplemented  by  job  sequence 
within  each  group. 

The  power  supplies  were  divided  into  four  functional  groups,  as 
described  in  a handout  (Appendix  B,  Vol.  II).  It  seemed  that  a major 
difficulty  in  learning  to  work  with  the  various  power  supplies  was  the 
confusion  resulting  from  their  nine  locations,  and  each  location  supply- 
ing multiple  voltages.  The  grouping  is  booed  upon-  the  BITE  (built-in 
test  equipment)  their  location,  and  their  function  in  the  system.  The 
grouping  corresponds  rather  closely  with  the  basic  block  diagram,  of  the 
system  (Figure  24-2  in  TM  9-1430-533-12-2-2). 

As  an  overview,  the  instructor  demonstrated  v.’hcre  checks  were  to  be 
made  on  each  group  of  power  supplies,  located  the  Indlviducl  power  sup- 
plies within  the  group,  and  showed  slides  of  them.  Then,  as  each  power 
supply  group  was  discussed  in  detail,  the  color  photos  of  that  group  were 
to  be  hung  near  their  locations  on  the  large  photos  and  the  photos  from 
the  previously  discussed  group  removed  so  as  to  distinguish  sharply  which 
group  was  being  discussed. 

In  both  experimental  classes  the  instructor  apparently  had  some  dif- 
ficulty in  hanging  and  removing  the  photos  of  the  power  supplies  according 
to  plan.  Sometimes  pictures  of  previously  discussed  power  supplies  would 
be  left  in  place.  Sometimes  the  pictures  within  a group  were  hung  one- 
by-one  rather  than  all  at  once.  Therefore,  tka  groups  may  have  been  less 
sharply  delineated  than  Intended  in  the  lesson  plan. 


* Volume  II  available  from  Army  Research  Institute. 


RF  Generation,  Arc  Detection,  and  Modulator  Bias  (8  Periods).  In 
this  presentation  the  Instructional  strategy  was  based  upon  the  under- 
lying causal  sequence  (as  was  the  conventional  lesson  plan)  supplemented 
by  visuals  showing  the  physical  components.  In  this  Instance  the  Master 
Oscillator  generates  the  RF  energy  Into  a waveguide,  which  Is  a rectangu- 
lar metal  tube.  The  energy  passes  through  various  components  that  envelop 
or  abut  the  rectangular  waveguide;  these  components  monitor,  rotate, 
modulate,  or  otherwise  process  the  RF  energy.  These  are  represented 
schematically  In  Figure  24-17  of  TM  9-1430-533-12-2-2.  In  performing  FIP, 
the  student  must  relate  these  schematics  to  the  equipment.  One  of  the 
difficult  things  to  teach  about  the  RF  Generation  was  that  the  process 
could  be  monitored  In  only  a few  places,  although  this  generally  was 
rather  obvious  from  the  physical  appearance  (but  not  from  the  schematics) . 
The  points  for  monitoring  show  up  as  test  jacks,  crystal  mounts  for  BITE, 
or  other  distinctive  components.  After  the  RF  Generation  chain  was  de- 
scribed completely,  the  specialized  circuits  for  Arc  Detection  and  Modu- 
lator Bias  were  described  and  Illustrated. 


On  Station  (PE  1) 


Tape/sllde  presentations  demonstrating  checks  and  adjustments  were 
developed  In  multiple  copies.  (One  presentation  was  for  Energizing  and 
Control  Circuits,  one  for  AC  and  DC  Power  Distribution,  and  one  for  RF 
Generation,  Arc  Detection,  and  Modulator  Bias,  although  the  latter  was 
not  part  of  the  experimental  units  of  instruction.)  Each  tape/slide 
demonstration  was  shown  once  to  all  students  In  another  room.  Each  demon- 
stration consisted  of  reading  the  check  manual  with  the  slides  demonstra- 
ting jrierformance  of  the  procedures. 

A ever ies  of  FIP  problems  was  developed  for  use  with  the  color  photos 
and  givsv^i  to  each  pair  of  students.  The  original  series  of  seven  prob- 
lems (for  class  75-7)  was  expanded  to  twenty  problems  for  class  75-8. 

The  directions  and  problems  are  given  In  Appendix  C (Vol.  II).  These 
problems  present  some  critical  information  and  approaches  to  FIP.  How- 
ever, there  were  difficulties  in  administering  these  problems  so  that 
effective  practice  conditions  were  assured.  First,  there  was  no  way 
devised  to  insure  that  the  students  actually  performed  the  problems  con- 
scientiously. Also,  there  was  no  way  to  Insure  that  the  students  referred 
to  the  color  photos  when  equipment  was  Involved  In  some  step.  Special 
evaluation  devices  and  techniques  that  were  considered  (such  as  an  answer 
sheet  on  which  erasures  reveal  the  correctness  or  error  of  each  response) 
seemed  to  entail  more  drawbacks  than  advantages.  The  resulting  Instruc- 
tional procedure  appeared  to  be  rather  haphazard. 


INSTRUCTOR  SELECTION  AND  PREPARATION 


The  Instructor  for  the  experimental  classes  was  the  instructor 
routinely  assigned  to  teach  class  75-7.  The  instructional  strategies 
were  explained  to  him  in  a briefing  which  emphasized  visual  simulation 
and  Job  sequencing.  The  detailed  methods  to  be  followed  were  described 
in  lesson  plans  (see  Appendix  A,  Vol.  II),  prepared  in  the  format  the 
instructor  was  used  to.  The  new  methods  were  practiced  in  a dry  run, 
with  occasional  comments,  coaching,  and  encouragement  by  the  Project 
Director.  The  Lesson  Plans  in  Appendix  A (Vol.  II)  reflect  minor  cor- 
rections and  adjustments,  which  were  requested  by  the  Instructor.  A 
second  series  of  dry  runs  was  conducted  similarly  in  preparation  for 
class  75-8. 


CRITERION  TESTING 


Routine  Examinations 


The  regular  examination  over  the  first  four  experimental  blocks  of 
instruction  was  one  troubleshooting  problem.  The  student  was  graded 
not  only  on  whether  he  found  the  fault,  but  also  on  the  procedure  he 
used.  Such  testing  was  insufficient  for  research  purposes  for  several 
reasons.  First,  the  testing  was  designed  to  discriminate  at  the  lower 
end  of  the  distribution,  in  order  to  determine  whether  a student  would 
pass  or  fall,  and  hence,  the  scores  tended  to  pile  up  at  the  upper  end 
of  the  distribution,  with  the  typical  student  getting  a near-perfect 
score.  Consequently,  these  exams  did  not  provide  much  room  for  improve- 
ment for  the  average  or  superior  student,  so  they  h.ave  limited  power  for 
detecting  differences  in  experimental  trainin'*  methods.  Also,  the  prob- 
lems were  not  standardized;  even  within  a test,  an  examiner  would  change 
problems  if  he  had  reason  to  suspect  that  the  students  who  had  taken  the 
exam  were  telling  the  others  about  it.  Finally,  a single  problem  does 
not  provide  enough  Information  to  evaluate  a program. 


Experimental  Examinations 


For  the  above  reasons,  an  experimental  testing  program  was  Instituted. 
The  normal  testing  program  was  scheduled  for  an  eight-period  block  of 
instruction  during  which  the  students  were  tested  individually.  After 
taking  their  official  examination,  each  student  in  both  experimental  and 
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control  classes  was  given  another  examination.  The  examiners  were 
regular  Army  enlisted  Instructors  assisted  by  contractor  personnel  who 
kept  time  and  helped  maintain  uniformity  of  procedure. 

The  experimental  examination  procedure  consisted  of  three  parts:  a 

Practical  Examination  on  the  equipment,  a Written  Multiple  Choice  Exam- 
ination, and  an  Attitude  Questionnaire  (the  corresponding  forms  are  in 
Appendices  D,  E,  and  F respectively,  Vol.  II). 


Praotiaal  Examinations.  During  the  Practical  Examination  the  equip- 
ment was  turned  off  and  the  students  actually  manipulated  switches  and 
pointed  to  dials,  describing  what  they  would  expect  normally.  The  first 
Item  was  to  perform  all  relevant  checks  and  adjustments,  followed  by  two 
Fault  Isolation  problems  (one  for  Energizing  Circuits,  one  for  AC  and  DC 
Power  Supplies).  Directions  are  in  Appendix  D,  Vol.  II.  For  the  checks 
and  adjustments,  each  student  was  given  a maximum  of  10  minutes  to  complete 
Tables  3-8,  3-9,  and  3-10  of  TM  9-1430-533-12-1.  This  resulted  in  rank 
order  data,  depending  upon  how  many  steps  they  completed,  or  if  they  com- 
pleted the  entire  procedure,  how  fast  they  completed  it.  The  number  of 
errors  were  also  noted.  For  the  Fault  Isolation  problems,  each  student 
was  allowed  five  minutes  for  each  of  the  two  problems.  The  Instructor 
would  provide  feedback  verbally  at  each  step.  One  problem  concerned  an 
Interlock  (In  the  Energizing  and  Control  Circuits)  and  the  other  involved 
a defective  Power  Supply  (from  the  lessons  on  AC  and  DC  Power  Distribution). 
The  resulting  data  are  whether  each  student  successfully  completed  the 
problem  in  the  5 minutes  allowed,  and  the  number  of  errors  made. 

On  the  Transmitter  RF  Generation,  Arc  Detection  and  Modulator  Bias 
Circuits,  the  testing  was  done  In  conjunction  with  another  Practical 
Examination  later  in  training.  That  experimental  testing  consisted  of 
one  representative  check  procedure  and  one  relevant  problem  in  trouble- 
shooting. For  the  check  procedure,  each  student  was  given  8 minutes  to 
complete  Step  19  of  Table  3-11.  Tills  yielded  rank  order  data,  plus  a 
tally  of  errors,  as  in  the  earlier  check  procedure.  For  the  Fault  Isola- 
tion problem,  each  student  was  allowed  7 minutes,  and  it  was  noted  whether 
the  problem  was  completed  successfully,  and  the  number  of  errors  made. 


Quizzes.  During  the  same  examination  period  that  each  class  was 
given  their  Practical  Examinations,  they  were  also  given  short  quizzes 
(Appendix  E,  Vol.  II).  The  items  in  these  quizzes  were  designed  to 
Involve  some  critical  element  of  job  performance.  One  quiz  covered 
Energizing  and  Control  Circuits.  Class  75-5  took  a completion  form  of 
this  test,  which  proved  difficult  to  score  objectively;  hence,  they  were 
not  included  in  the  analysis  of  quiz  scores  for  Energlizlng  and  Control 
Circuits.  The  quiz  on  Energizing  and  Control  Circuits  consisted  of 
Identifying  the  nine  locations  of  the  power  supplies. 
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Opinion  Queationnairee.  To  supplement  the  test  data,  questionnaires 
were  administered  to  all  classes  (Appendix  F,  Vol.  II) , in  which  the 
students  rated  the  general  effectiveness  of  Instruction.  One  section, 
administered  to  the  Experimental  Group  only,  asked  for  absolute  judgments 
of  the  effectiveness  of  the  visual  aids  and  FIP  problems. 
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ANALYSIS 


COMPARABILITY  OF  GROUPS 


The  conparablllty  of  the  Experimental  and  Tontrol  Groups  was  first 
established  by  comparing  them  on  the  E.L.  Score  of  the  Army  Classifica- 
tion Battery  (ACL)  by  which  they  were  qualified  for  the  24C20  POI. 


DATA  SPECIALLY  COLLECTED 


Practical  Examination  Scores 


Checks  and  Ad.iustivtnts . Here  the  data  consisted  of  rank-ordered 
scores,  based  upon  speed  In  completing  the  checks  (differences  tested 
by  Mann-i'Ihltney  "U")  and  the  number  of  errors  In  performing  them 
(differences  tested  using  x^) • 


Troubleshooting.  There  were  two  FIP  problems  administered  on  the 
first  examination  day,  and  one  on  the  second.  It  was  noted  whether 
each  person  successfully  completed  each  problem  In  the  time  allotted, 
and  the  number  of  errors.  The  statistic  was  used  to  test  signifi- 
cance of  both  completion  and  errors. 


Quizzes 


Differences  between  group  means  for  each  of  the  three  short  quizzes 
were  tested  using  the  "t"  test. 


Opinion  Questionnaires 


Only  descriptive  data  were  computed  for  the  student  questionnaires. 


RESULTS 


COMPARABILITY  OF  GROUPS 


• On  the  E.L.  Score  of  the  Army  Classification  Battery  (for  those  scores 
available),  the  mean  for  classes  75-5  and  75-6  (Control  Group)  was  112.25 
(n  ■ 39),  and  for  classes  75-7  and  75-8  (Experimental  Group)  was  112.77 
(n  - 27) . The  very  small  difference  wa*:.  well  within  the  range  that  would 
be  expected  by  chance  alone.  It  Is  assumed,  therefore,  that  the  groups 
were  comparable. 


ROUTINELY  ADMINISTERED  EXAMINATIONS 


The  results  of  the  first  school  practical  examination  were  highly 
skewed,  with  the  majority  scoring  above  90%.  The  median  score  (50th 
percentile)  for  classes  75-5  and  75-6,  taken  together  was  96  out  of  100, 
compared  with  95  for  the  experimental  classes  (75-7  and  75-8).  The 
difference  was  not  significant. 


EXPERIMENTAL  PRACTICAL  EXAMINATION 


Checks  and  Adjustrnents 


The  classes  were  compared  on  rank-order  data,  derived  from  the  speed 
with  which  they  performed  the  procedures  specified  in  Tables  3-8,  3-9, 
and  3-10  ■ (TM  9-1A30-533-12-1) . An  appropriate  statistic  is  the  Mann- 
Whltney  "U"  (Siegel,  1956,  pp.  116-126)  which  tests  whether  one  group 
ranks  significantly  higher  than  another  when  the  two  are  combined  in  a 
rank-order  distribution. 

When  the  Experimental  Group  (classes  75-7  and  75-8)  was  compared  with 
the  Control  Group  (classes  75-5  and  75-6) , the  Experimental  classes  scored 
significantly  better  (z  ■ 5.89,  p < .001).  Only  four  men  in  the  Experi- 
mental Group  scored  below  the  median  for  the  Control  Group;  no  one  in  the 
Control  Group  scored  above  the  median  for  the  Experimental  Group.  This 
was  a cumulative  effect  over  the  first  four  experimental  blocks  of  instruc- 
tion, and  the  separate  effects  of  particular  blocks  could  not  be  isolated. 
The  Experimental  Group  also  tended  to  make  significantly  fewer  errors 
(Table  1,  x\  ■ 11.08,  p < .01). 
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On  the  second  Practical  Examination  covering  RF  Generation  (Step  19 
of  Table  3-11,  TM  9-1A30-533-12-1)  almost  all  of  the  students  did  finish 
within  the  time  limit  (8  minutes)  so  the  data  were  rank-order  based  upon 
time  to  finish.  The  Experimental  Group  was  significantly  faster  (z-2.28, 
p < .05).  This  difference  was  much  smaller  than  the  corresponding  one  In 
the  previous  test  on  checks  and  adjustments,  although  still  significant, 
statistically.  The  median  score  for  the  Experimental  Group  was  near  the 
70th  percentile  for  the  Control  Group.  The  differences  In  number  of  errors 
(Table  1)  was  not  significant. 


Troubleshooting 


During  the  first  Practical  Examination  each  student  also  performed 
two  troubleshooting  problems  on  the  equipment  with  the  Instructor  verbally 
supplying  feedback.  (See  Appendix  D,  Vol.  II,  for  details.) 

On  the  first  problem  3A  out  of  39  students  In  the  Experimental  Group 
completed  the  problem  successfully  while  only  20  out  of  39  In  the  Control 
Group  (classes  75-5  and  75-6)  completed  it.  (Results  in  Table  2.)  This 
difference  was  significant  statistically  (x^  “ 13.2,  p < .001). 

On  the  second  problem  the  Experimental  Group  also  performed  better: 

26  out  of  39  completed  it,  compared  with  11  out  of  39  in  the  Control  Group 
(x^i  ■ 10.56,  p < .001).  They  also  made  significantly  fewer  errors  (x^,  = 
15.92,  p < .001). 

On  the  problem  in  the  RF  Generation  system  (Table  3) , the  Experimental 
Group  tended  to  finish  somewhat  more  often  (x^,  ■ 5.3,  p < .05),  although 
there  was  no  significant  difference  In  the  number  of  errors  made. 


Quizzes 


On  the  quiz  covering  Encrgiz ing_and  Control  Circuits,  the  Experimental 
Group  scored  significantly  better  (X  “ 5.23  vs.  A. 22,  t ■ A. 372,  p < .001) 
than  the  Control  Group  (class  75-6  only,  as  noted  previously  under  METHOD). 
Similarly,  on  the  quiz  on  locations_of  the  power  supplies,  the  Experimental 
Group  scored  significantly  better  (X  ■ 7.82  vs  5.9A,  t • A. 67,  p < .001). 

On  the  quiz  covering  RF  Generation,  Arc  Detection,  and  Modulation  Bias, 
the  Experimental  Group  also  tended  to  score  somewhat  better  (X  9.35  vs. 
7.A2,  t - 3.2A,  p < .01). 
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TABLE  1 


PRACTICAL  EXAMINATION, 


RS  ON  CHECKS  AND  ADJUSTMENTS 


0 

1 

2 or  more 

Experimental  Group 

14 

9 

16 

Control  Group 

3 

7 

29 

Experimental  Group 
Control  Group 


Experimental  Group 


Control  Group 


TABLE  2.  PERFORMANCE  ON  FAULT  ISOLATION  PROBLEMS 
IN  FIRST  EXAMINATION  PERIOD 


Problem  1 (Interlock) 


Experimental 

DID  NOT 
COMPLETE 
IN  TIME 

COMPLETED 
IN  TIME 

ERRORS 

m 

1 

2 or  more 

5 

34 

27 

10 

2 

Control 

19 

20 

11 

14 

14 

x^i  «=  13.2,  p < .001  x^2  “ 16.4,  p < .001 


Problem  2 (+100  VDC  Power  Supply) 


Experimental 

DID  NOT 
COMPLETE 
IN  TIME 

COMPLETED 
IN  TIME 

ERRORS 

1 

0 

1 

2 or  more 

13 

26 

15 

20 

4 

Control 

28 

11 

6 

13 

20 

X^i  = 10.6,  p < .001  x^2  “ 15.9,  p < .001 
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TABLE  3.  PERFORMANCE  ON  FAULT  ISOLATION  PROBLEMS 
IN  SECOND  EXAMINATION  PERIOD 


Problem  1 (Crystal  Detector) 


Experimental 

DID  NOT 
COMPLETE 
IN  TIME 

COMPLETED 
IN  TIME 

ERRORS 

0 

1 

2 or  more 

3 

35 

7 

12 

19 

Control 

11 

28 

5 

5 

29 

x^i  * 5.3,  p < .05  x^2  ® 5.3,  n.s. 


Opinion  Questionnaires 


All  classes  held  rather  favorable  attitudes  toward  the  instruction; 
differences  between  Experimental  and  Control  Groups  were  negligible  (see 
Appendix  F,  Vol.  II).  These  data  are  summarized  in  Table  A.  Attitudes 
of  the  experimental  classes  tov?ard  the  particular  experimental  practices 
were  generally  rather  favorable.  On  use  of  media  in  the  classroom  all  but 
one  of  the  students  rated  it  "somevdiat  helpful"  or  "a  definite  help". 
Similarly,  on  the  paper  troubleshooting  problems,  all  but  two  students 
rated  them  "somewhat  helpful",  "helpful",  or  "very  helpful".  However, 
attitudes  toward  the  tape/s.lide  demonstrations  of  checks  and  adjustments 
were  somewhat  less  favorable;  twelve  of  tlie  students  rated  them  in  the 
lower  categories  of  "of  little  help"  or  "a  waste  of  time". 


21 


TABLE  4. 


QUESTIONNAIRE  SUMf^ARY 


The  following  data  are  mean  check  ratings  of  how  interesting  and 
effective  the  instruction  was,  based  upon  a nine-point  scale  with  9 
being  most  favorable.  Ratings  are  averaged  across  subjects  and  across 
the  four  questions. 


EXPERIMENTAL 

CONTROL 

CLASSROOM 

5.7 

6.4 

ON  STATION 

7.2 

7.1 

(See  also  Appendix  F,  Vol.  II.) 
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DISCUSSION  AND  INTERPRLTATION 


The  experimental  instructional  strategies  employing  visual  simulation, 
In  general,  produced  significantly  greater  ability  to  perform  Job  proce- 
dures in  a specified  period  of  testing  time,  with  significantly  fewer 
errors,  and  also  resulted  In  significantly  better  quiz  scores.  These 
results  were  achieved  by  a typical  Instructor  of  USAADS  Primary  Instructors, 
with  preparation  Involving  a briefing,  lesson  plan,  and  a "dry  run"  for 
practice,  which  Is  normal  preparation  for  any  new  method  of  Instruction. 

The  experimental  Instructional  methods  also  did  not  require  extensive 
devices  and  they  were  practicable  In  the  USAADS  Instructional  situation. 

The  effects  of  the  various  practices  are  thoroughly  confounded  In  the 
results.  However,  based  upon  observations  of  the  process,  it  seems  reason- 
able to  attribute  most  of  the  results,  tentatively,  to  the  classroom  acti- 
vities. These  practices  affected  many  hours  of  Instruction  very  noticeably. 
The  paper  troubleshooting  and  tape/slide  deuonstratlons  of  checks  and  ad- 
justments affected,  at  moat,  only  a few  minutes  of  instruction.  Also, 
performance  on  the  problems  was  not  subject  to  effective  monitoring  prac- 
tices, so  there  was  little  Incentive  beyond  intrinsic  interest  in  the 
problems,  and  a rather  tenuous  relationship  to  eventual  test  performance. 
Yet,  the  students  reacted  favorably  to  the  problems,  according  to  the 
opinion  data,  so  the  few  minutes  spent  on  them  involves  no  drawbacks  and 
may  well  be  helpful. 

There  is  a strong  tradition  In  electronics  training  that  relegates 
theory  to  the  classroom  and  performance  to  the  actual  equipment.  During 
the  development,  several  instructors  commented  that  actual  performance 
Is  not  the  sort  of  thing  that  Is  taught  in  the  classroom,  and  there  wan 
sometimes  confusion  concerning  whether  we  were  discurslng  PE  3 (classroom) 
or  PE  1 (work  with  the  equipment).  Thus  theory  and  practice  have  come  to 
be  Increasingly  separate  (in  spite  of  official  directives  to  the  contrary). 
The  Instructional  strategies  using  visual  simulation  are  based  upon  quite 
a different  point  of  view;  that  theory  and  practice  are  quite  compatible, 
that  they  can  and  should  be  taught  together,  and  that  visualization  is 
actually  an  Important  tool  for  thinking. 
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